ERC as a Trendsetter?

Measuring Similarity in the Objectives of Funded Research in Europe

Radim Hladík,1 Marc Bertin,2 Chérifa Boukacem-Zeghmouri2

1 Centre for Science, Technology, and Society Studies (CSTSS)
Institute of Philosophy, Czech Academy of Sciences
<hladik@flu.cas.cz>, <@hlageek@bsky.com>

2 ELICO
Université Claude Bernard Lyon 1
<marc.bertin@univ-lyon1.f>,
<cherifa.boukacem-zeghmouri@univ-lyon1.f>

Excellence in science

  • scientific excellence is as ubiquitous in science policy and research evaluation as it is challenging to define
  • often a “know it when you see it” concept
  • in ever-scarcer science funding, excellence - of proposals and/or applicants - is used as a fundamental criterion for funding decisions

How can we research excellence?

  • from philosophy of science to sociology of science
    • even if excellence lacks objective definition, we can work with a practical definition
      • excellent science is what ERC think it is
    • which sociologically translates to:
      • excellent science is what ERC funds

Data

A convenience sample from European funding agencies providing project metadata including abstracts in English.

9 agencies / 8 countries + transnational ERC

  • European Research Council (ERC), European Union
  • Agence nationale de la recherche (ANR), France
  • Deutsche Forschungsgemeinschaft (DFG), Germany
  • Fonds Wetenschappelijk Onderzoek (FWO), Belgium
  • Grantová agentura České republiky (GAČR), Czech Republic
  • Norges Forskningsråd (NFR), Norway
  • Schweizerischer Nationalfonds / Fonds National Suisse (SNSF), Switzerland
  • UK Research and Innovation (UKRI), United Kingdom
  • Vetenskapsrådet (VR), Sweden

Analysis dataset

  • automated language checks
  • aligning years (some agencies publish year of the call, some date of the project start, or both)
  • project duration >= 3 years & <= 6 years
  • 98455 project abstracts included in the analyses
Year ANR GACR DFG ERC NFR FWO SNSF UKRI VR
2009 238 670 317 273 53 129 612 1646 995
2010 753 470 439 483 41 157 729 1587 894
2011 727 442 428 695 70 140 669 1353 872
2012 709 459 682 780 43 148 781 1808 1065
2013 596 404 1026 881 41 219 773 1920 1019
2014 761 338 1237 902 45 182 741 1666 821
2015 767 538 1280 935 58 192 778 1631 791
2016 953 577 1468 928 50 220 696 1600 739
2017 1059 732 1451 935 44 259 635 1499 634
2018 1129 585 1666 1001 22 329 530 1394 718
2019 1150 681 2237 918 22 330 556 1324 720
2020 1211 544 2552 908 27 346 488 1181 720
2021 1770 329 2688 973 106 328 585 1485 739
2022 1698 475 2419 1043 11 328 585 1583 643

Methods

  • descriptive
  • abstract embeddings obtained via SPECTER2 (2023 update)
    • BERT-type language model of scientific texts trained over a citation graph
  • 2D dimension reduction via UMAP
  • non-ERC projects assigned to the ERC panel of the closest ERC project
    • validated by reassignment of ERC projects (M = 0.96, SD = 0.03, range [0.85, 1.00])
  • RQ1: measuring mean pairwise similarity of non-ERC projects to ERC projects within panel-year combinations
  • RQ2: measuring the spread of projects in the 2D embedding space via convex hull area

UMAP projection of projects

Dots signify average coordinates of ERC panels (true and estimated).

cf. maps of science - Rafols 2025; Hladík & Renisio 2025; Klavans & Boyack 2009

Density of projects in the UMAP projection

Preliminary results

  • quantitative evidence of growing similarity of nationally funded projects to ERC
    • caveat: causality cannot be determined
      • is ERC setting the trend or is it following a confounder along with other funders?
  • the assimilation process is slow, i.e. a long-term perspective is important
  • the epistemic scope appears unrelated to the assimilation process
    • rather a function of the number of projects funded (i.e. budget)

Discussion

  • comparative lense: research on research funding benefits from transcending single-funder or single-country approaches and taking heed of transversal patterns (Aagaard et al. 2021)
  • data on research funding: lack of metadata standards and data sharing
  • excellence
    • an emerging norm for presenting research objectives to peers
    • competitive research funding as a channel for dissemination of the norm
    • possibly performative (in both senses - self-actualizing and without substance)
    • stratification among PIs - high ERC prestige
  • policy: is the growing standardization of excellence a desirable policy goal?

Acknowledgements

This work has been funded by a grant from the Programme Johannes Amos Comenius under the Ministry of Education, Youth and Sports of the Czech Republic, CZ.02.01.01/00/23_025/0008711.